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pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found 
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in 
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web 
server ( http://webapp.etsi.org/IPR/home.asp) . 

Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee 
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web 
server) which are, or may be, or may become, essential to the present document. 



Foreword 

This Technical Specification (TS) has been produced by ETSI Technical Conmiittee Speech and multimedia 
Transmission Quality (STQ). 

The present document is part 7 of a multi-part deliverable covering the QoS aspects for popular services in GSM and 
3G networks, as identified below: 

Part 1 : "Identification of Quality of Service criteria" ; 

Part 2: "Definition of Quality of Service parameters and their computation" ; 

Part 3: "Typical procedures for Quality of Service measurement equipment" ; 

Part 4: "Requirements for Quality of Service measurement equipment" ; 

Part 5 : "Definition of typical measurement profiles" ; 

Part 6: "Post processing and statistical methods"; 

Part 7: "Network based Quality of Service measurements". 

Part 1 identifies QoS aspects for popular services in GSM and 3G networks. For each service chosen QoS indicators are 
listed. They are considered to be suitable for the quantitative characterization of the dominant technical QoS aspects as 
experienced from the end-customer perspective. 

Part 2 defines QoS parameters and their computation for popular services in GSM and 3G networks. The technical QoS 
indicators, listed in part 1, are the basis for the parameter set chosen. The parameter definition is split into three parts: 
the abstract definition which contains a generic description of the parameter, the abstract equation and the respective 
trigger points. Only measurement methods not dependent on any infrastructure provided are described in the present 
document. The harmonized definitions given in the present document are considered as the prerequisites for comparison 
of QoS measurements and measurement results. 

Part 3 describes typical procedures used for QoS measurements over GSM and 3G networks, along with settings and 
parameters for such measurements. 

Part 4 defines the minimum requirements of QoS measurement equipment for GSM and 3G networks in the way that 
the values and trigger points needed to compute the QoS parameter as defined in part 2 can be measured following the 
procedures defined in part 3. Test equipment fulfilling the specified minimum requirements will allow to perform the 
proposed measurements in a reliable and reproducible way. 

Part 5 specifies test profiles which are required to enable benchmarking of different GSM or 3G networks both within 
and outside national boundaries. These profiles are necessary for comparing "like-for-like" performance in case that a 
specific set of tests is carried out by different customers. 
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Part 6 describes procedures to be used for statistical calculations in the field of QoS measurement of GSM and 3G 
networks using probing systems. 

Part 7 describes how Quality of Service measurements should be done inside the network without direct access to the 
end point terminal. 

Introduction 

Measurements of service performance can be done either with an end-point test measurement tool, ether stationary or 
mobile drive test, or inside the network itself. The measurements should always be done from an end-user perspective, 
independent of the measurement point. Obviously a measurement done inside the network might not give exactly the 
same result as a measurement done in the end-point with a test tool. 

However, also the network measurement can give valuable information about service performance as the end-user 
perceives it. It is also possible to take more samples of the service performance with a network based measurement than 
with an end-point test tool. The service performance measurements discussed presented in the present document shall 
all be based on standardized protocols and interfaces. 

The quality of service parameters in TS 102 250-2 [1] are initially specified for an end-point test tool measurement 
scenario. However, the parameters can be reused for network based measurements with some limitations and minor 
changes. 
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Scope 



The present document specifies how the quaUty of service parameters, Hsted in TS 102 250-2 [1], should be used for 
measurements done inside a network, in contrary to measurements done in the end-point with a test tool. A test tool can 
be either stationary or a drive test tool. The measurements of the QoS parameters according to the present document 
should be done using standardized interfaces and protocols. This is done to ensure that all measurements in a 
multi-vendor network can be compared. 



2 References 

References are either specific (identified by date of publication and/or edition number or version number) or 
non-specific. 

• For a specific reference, subsequent revisions do not apply. 

• Non-specific reference may be made only to a complete document or a part thereof and only in the following 
cases: 

if it is accepted that it will be possible to use all future changes of the referenced document for the 
purposes of the referring document; 

for informative references. 

Referenced documents which are not found to be publicly available in the expected location might be found at 
http://docbox.etsi.org/Reference . 

NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee 
their long term validity. 

2.1 Normative references 

The following referenced documents are indispensable for the application of the present document. For dated 
references, only the edition cited applies. For non-specific references, the latest edition of the referenced document 
(including any amendments) applies. 

[1] ETSI TS 102 250-2: "Speech and multimedia Transmission Quality (STQ); QoS aspects for 

popular services in GSM and 3G networks; Part 2: Definition of Quality of Service parameters and 
their computation" . 

[2] Void. 

[3] ETSI TS 126 234: "Universal Mobile Telecommunications System (UMTS); LTE; Transparent 

end-to-end Packet-switched Streaming Service (PSS); Protocols and codecs (3GPP TS 26.234)". 

[4] ETSI TS 126 346: "Universal Mobile Telecommunications System (UMTS); LTE; Multimedia 

Broadcast/Multicast Service (MBMS); Protocols and codecs (3GPP TS 26.346)". 

[5] ETSI TS 126 1 14: "Universal Mobile Telecommunications System (UMTS); LTE; IP Multimedia 

Subsystem (IMS); Multimedia telephony; Media handling and interaction (3GPP TS 26.114)". 

[6] ITU-T Recommendation P. 564: "Conformance testing for voice over IP transmission quality 

assessment models". 

[7] ITU-T Recommendation P.862.1: "Mapping function for transforming P. 862 raw result scores to 

MOS-LQO". 

[8] Void. 
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[9] 



ETSI TS 124 008: "Digital cellular telecommunications system (Phase 2+); Universal Mobile 
Telecommunications System (UMTS); LTE; Mobile radio interface Layer 3 specification; Core 
network protocols; Stage 3 (3GPP TS 24.008 Release 8)". 



2.2 



Informative references 



The following referenced documents are not essential to the use of the present document but they assist the user with 
regard to a particular subject area. For non-specific references, the latest version of the referenced document (including 
any amendments) applies. 

Not applicable. 



3.1 



Definitions and abbreviations 



Definitions 



For the purposes of the present document, the following terms and definitions apply: 
drive test tool: end-point test tool which is designed to be moved around, i.e. by walking or driving a car 
end-point test tool: typically especially designed mobile which uses active test calls to collect measurements 
stationary tool: end-point test tool which is installed in a fixed location 



3.2 



Abbreviations 



For the purposes of the present document, the following abbreviations apply: 



3G 

3GPP 

ACK 

FTP 

Gn 

GPRS 

GSM 

GSN 

lub 

MO 

MT 

PCO 

PoC 

POR 

QoE 

QoS 

RNC 

RRC 

SGSN 

TCP 

UE 



3^*^ Generation 

Third Generation Partnership Project 

Acknowledgement 

File Transfer Protocol 

Interface between GSN nodes 

General Packet Radio Service 

Global System for Mobile communications 

GPRS Support Node 

Interface between RNC and Node B 

Mobile-Originated 

Mobile-Terminated 

Point of Control and Observation 

Push to talk over Cellular 

Point of Recording 

Quality of Experience 

Quality of Service 

Radio Network Controller 

Radio Resource Control 

Serving GPRS Support Node 

Transmission Control Protocol 

User Equipment 
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4 Network Measurement Basics 

4.1 Point of Control and Observation (PCO) 

The Point of Control and Observation (from now on called "point of observation" or PCO) is the location where the 
measurement is actually performed. The location can be either inside the network or in the end-point. The 
measurements should be done using standardized interfaces and protocols. 

Possible points of observation for QoS parameters covered in the present document are: 

• Inside nodes in the network (RNC, base station, switch, etc.) 

• Observations in the terminal: 

End-point test tool; or 

Measurements that are reported back from the terminal to the network 



PCO in mobile 

(can be multiple PCO) 



PCO in network node 
or at network node 
interface 



POR in a network node and 
data is reported from a PCO in 
a terminal 




Figure 1: Points of Observation, Points of Recording and Measurement Reporting 



4.2 Point of Recording (POR) 



The point of recording (POR) is where the QoS parameters are recorded. The POR can be the same as the PCO or 
another point inside the terminal or the network. If the PCO and the POR are not the same, the measurement data must 
be reported from the PCO to the POR. Examples of such reporting are described in annex A. 



Measuring QoS Parameters in the Network 



5.1 



General Overview 



The quality of service measurements should be done as much as possible in the same way inside the network as they are 
done in the end-point with a test tool. Many types of measurements can be done with the same trigger points (for 
instance, the reception of a certain protocol message) independent of the point of measurement, but the measurement 
result might differ slightly depending on where in the call or transmission chain the PCO is located. 
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5.2 Service Accessibility QoS Parameters 

Accessibility QoS parameters reflects the ability to initiate or intentionally terminate a connection or a service. The 
parameters can be divided into the following groups: 

• Failure parameters: Reflects the outcome of attempts to initiate a connection or a service. For mobile- 
originated (MO) cases the network might sometimes be unaware of some of the attempts, and network-based 
parameters can be expected to give a slightly more positive view of the network condition, as compared to the 
corresponding endpoint-test parameters. 

For mobile-terminated (MT) initiation attempts the network is normally fully aware of these, and network- 
based parameters should therefore correspond well to the same parameters as measured by an endpoint test 
tool (assuming that MT initiation attempts are known and controlled by the endpoint test tool). 

As most initiation procedures require a successful two-way communication during the initiation phase, the 
accessibility parameters measured for MO and MT endpoint test calls should normally not differ too much, 
and thus the network-based parameters for the MT case can be seen as a good approximation of the total 
network state. 

• Time parameters: Reflects the time needed to initiate a connection or a service. As these parameters are only 
defined for successful attempts the network can see the message flow, and can measure the time elapsed to 
initiate the connection or the service. 

The difference in parameter values as compared to the corresponding endpoint test measurements depend on 
where in the network the time measurements are done, but normally the time elapsed in the radio link and the 
processing time in the mobile are not included in the network-based parameters, making them more optimistic. 

If the excluded radio delay is stable or small compared to the total delay, the network-based measurements can 
still give a good picture on the state of the network. 

The estimated value of the excluded delay parts (for instance the radio delay) should be added to (or noted 
together with) the measured time parameter. 

5.3 Service Retainability QoS Parameters 

Retainability QoS parameters reflect the ability to retain, or keep a service up and running. Typical examples of 
retainability parameters are cut-off ratio and session failure ratio. Retainability parameters can be measured on the end- 
point but in general also inside the network. Measurements inside the network do not in general need any additional 
measurement data from the end-point. 

5.4 Service Integrity QoS Parameters 

Service integrity QoS parameters reflect the quality of a service that has been successfully set up and is in use. As the 
integrity parameters are only measured for successfully connected services, the network will always be aware of the 
ongoing service, and can measure its performance. 

Depending on the type of service used different types of integrity parameters are calculated: 

• Media quality parameters 

• Response time parameters 

• Data rate parameters 
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5.4.1 Media Quality Parameters 



In an endpoint test scenario the media quality parameters can be measured in the mobile by using objective quality 
algorithms, for instance P. 862.1 [7] for speech quality measurements. For network-based quality measurements other 
methods are used: 

• Parameters related to the service quality can be measured at different nodes in the network, and translated to a 
service quality parameter by using a parametric or bitstream quality model, like P. 5 64 [6] or similar model for 
video and multimedia service. The resulting quality parameter will reflect the state at the measurement nodes, 
and be an estimate of the end-user quality. 

• Parameters related to the service quality can be measured in the mobile, and reported back to POR in a node in 
the network (see annex A). The calculation node combines the reported parameters with other relevant 
information collected from the network about the ongoing service, and calculates the service quality 
parameter. The resulting quality parameter will be closely correlated to the end-user experience. 

5.4.2 Response Time Parameters 

Some services are characterized by real-time events or real-time interaction between the users, and the response time 
needs to be short enough to provide a good service experience. A typical service example is PoC which is dependent on 
short delays between user input and system acknowledgement. 

The time elapsed between certain protocol events can normally be measured in different network nodes. Some parts of 
the response time (typically the radio part) will not be included in the network-based measurements, so the resulting 
response time parameters will only be an estimate of the end-user experience. 

However, if the radio delay is stable or small compared to the total delay, the network-based measurements can still 
give relevant information about the end-user experience. It might also in some cases be possible to make a separate 
measurement or estimate of the radio delay. 

Any estimated value of the excluded delay parts (for instance the radio delay) should be added to (or noted together 
with) the measured time parameters. 

5.4.3 Data Rate Parameters 

Data rate parameters, for services such as FTP, web browsing and email, can be measured at different nodes in the 
network. Normally these services are using acknowledged radio bearers, and the network-based data rate measurements 
should thus be closely correlated to the corresponding endpoint test measurements. 



6 Comparing Network and End-point Test 

IVIeasurements 

Except for the differences due to different PCO for end-point tests tools and network measurements also other factors 
will affect the result. For instance the customer behavior might not be the same, as in the following example: 

There is a radio network coverage problem for a series of tunnels in a newly built motorway area. Mobile users will 
frequently get their calls dropped when they pass this area on the motorway. Initially when the motorway opened the 
call drop rate measured in the network was high, since most of the calls are dropped. However, after some time the 
mobile users frequently passing this problem area will learn that the call will drop, and starts to avoid making calls 
when passing that point. The drop rate in the network will go down but the problem has actually not been solved. 

If the same area is also monitored by e.g. automatic end-point test equipment the drop rate will not go down, since the 
end-point test tool calling frequency is not reduced due to the coverage problem. 

The network measurement results and the end-point test tool measurements are both correct, yet they might have a very 
different drop rate. This illustrates that even if the same thing is measured in both cases, the results should not always be 
expected to be the same. 

Another important difference between end-point test tools and network measurements is the possibility to get a good 
geographical position for the end-point test tools data, while network data typically is limited to cell level resolution. 
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Thus even if network data in many cases can measure the same parameters, specific problems discovered in a cell might 
still need further support from endpoint testing to pinpoint where in the cell the problem occurs. 
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Annex A (informative): 

The concept of QoE reporting 



A QoE reporting mechanism has been standardized in 3GPP for PSS [3], MBMS [4] and MTSI [5]. The standards make 
it possible for the operator to activate QoE feedback report from the mobiles whenever PSS, MBMS or MTSI services 
are used. This makes it possible to closely follow relevant parameters from an end-user service quality perspective. 

The specific implementation differs slightly between the three cases, but has a common conceptual structure: 

• The operator can activate QoE reporting for a specific terminal, or for a statistical subset of the terminals. For 
instance, it might be enough to monitor only 10 % of the terminals to get a statistically good result, and thus 
saving some cell capacity due to lower amount of QoE reporting. 

• The operator can specify which parameters the terminal should report, and how often each parameter should 
be calculated. For instance, the terminal can be ordered to measure the amount of packet loss and rebuffering 
for a streaming service, or the frame loss and speech codec usage for an MTSI speech service. A typical 
parameter calculation length could be in the order of 8 seconds to 12 seconds, which corresponds to commonly 
used sequence lengths for subjective tests. 

• The operator can specify how often the measurements calculated by the terminal should be reported back to 
the system, for instance every 5th minute. This allows a more efficient feedback transmission, as several 
measurements calculated and buffered by the terminal are packed together before sending. Except for the 
periodic reports a final report is always sent after the service has ended, with the remaining buffered 
measurements. 

NOTE: While the QoE feedback reporting feature can be very useful for the operator, it is currently specified as 
"optional" in the 3GPP standards. This means that some terminals have implemented the feature, while 
others have not. Some terminals might have implemented the feature in the terminal platform, but only 
includes it in specific terminal models if the operator asks for it. 



ETSI 



13 



ETSI TS 102 250-7 VI .1.1 (2009-10) 



Annex B (informative): 

Examples of Network Based QoS Measurements 

B.1 Accessability and Retainability Parameters 

Figure B.l shows a subset of the protocol messages sent for a mobile-terminated call. The full sequence is described in 
TS 102 250-2 [1] under the clause "Telephony Service Non Accessibility". 



Terminal 




NodeB 




RNC 




1 

^ Paging "Terminating Conversational Call" 



© 




-^ 


1 

RRC Connection Request "Term. Conv. Call" 








1 

(lots of other signalling skipped here) 








1 

Alerting 




© 






1 

(lots of other signalling skipped here) 








1 

Disconnect 




© 








1 




^ 



Figure B.1 : Call setup and disconnect sequence for a mobile-terminated speech call 

This example measures the Telephony Service Non- Accessability for a circuit- switched call. The measurements are 
done on the lub interface close to the RNC. The starting trigger for the measurement can be either point 1 or point 2. If 
point 1 is chosen the measurements may include some terminals which are temporarily out of coverage, but it will 
include all call attempts. If point 2 is chosen, the measurements will only include terminals which have coverage, but 
might miss the call setup for some terminals which have basic signal strength coverage (i.e. they seem to have Radio 
Network Availability) but not enough to be able to send the RRC Connection Request. 

The end trigger for a successful call setup is point 3, where the Alerting message is sent to the terminal. If the end 
trigger is not reached the call attempt is counted as unsuccessful. This can for instance indicate that the terminal has 
sufficient signal strength to be able to use the signalling channel, but not enough to be able to set up the dedicated 
speech radio bearer. 

The time to make a successful call setup (Telephony Setup Time) can else be measured using the same example as 
above, where the time difference between the end trigger (point 3) and the start trigger (point 1 or 2) is measured. 

The retainability (Telephony Cut-Off Call Ratio) can be measured by observing the messages at point 3 and 4. A call is 
counted as being cut-off if the start trigger (point 3) is seen and the call is ended when the network (call control 
entity - the MSC) sends a DISCONNECT message to the terminal with a cause different to normal call ending. See 
TS 124 008 [9] for details about signaUng for "Call Clearing" in an UMTS network. 
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B.2 Media Quality Parameters 



In this example the media quaUty is estimated for the 3GPP PSS (Packet Switched Streaming Service) [3], where the 
parameters are measured in the terminal (PCO) and sent to the network (POR) using a the 3 GPP standardized QoE 
reporting. The steps involved in measuring the media quality are: 

• Initial service access initiated from the terminal (see figure B.2, point 1). 

• The media server sets up QoE reporting from terminals using the SDP and the RTSP protocols (point 2). The 
QoE setup is performed when the user wants to see a video or hear audio where the PSS service is used. The 
terminal acknowledges the QoE configuration (point 3). 

• The terminal measures the QoE parameters and sends periodic QoE reports to the media server (POR) during 
the PSS session (point 4). The QoE reports are sent using the RTSP protocol. 

• The QoE reports are used together with additional information from the server to calculate the perceived media 
quality. The media quality MOS is estimated using an objective parametric media quality model. 



Terminal (PCO) 


PSS Service Setup 


Media server 
(POR) 








© 
© 
© 
© 






QoE Setup 








QoE Setup Ack 








QoE Data every Nth Second 










^ 



Figure B.2: Media Quality measurement for tlie 3GPP PSS service 



B.3 Response Time Parameters 

In this example the HTTP IP-Service Setup Time QoS Parameter is measured at the SGSN node interface for a HTTP 
download of a single file from a server. The steps involved in the measurement are: 

• The HTTP session is set up between the terminal (client) and the HTTP server. 

• At a standardized interface at the SGSN the data packets for the HTTP session are inspected and the HTTP 
and TCP protocol headers are analysed. 

• The time when the "HTTP get" request from the HTTP client to the HTTP server leaves the SGSN on the Gn 
interface (the PCO and the POR) is measured. This is represented with the upper circle (point 1) in the 
sequence diagram for the SGSN node in figure B.3. 

• When the first data packet containing content is received at the SGSN interface the time of arrival is recorded. 
This is represented by the lower circle (point 2). 

• The Response Time is calculated by calculating the time difference between receiving the "HTTP get" 
command and receiving the first data packet containing content. 
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Terminal 



NodeB 



RNC 



SGSN 

(PCO and POR) 



GGSN 



HTTP file download request 

— \ — ^ — 

Handshake for HTTP download using TCP 



First data packet containing content 

— \ — ^ — 



Other data packets 



Last data packet containing content 

I CD 

HTTP transfer complete 



HTTP 
Server 



Figure B.3: Measuring HTTP download Mean Setup Time at standardized interface 

at the SGSN node 

Note that the measured time above does not include the time for the transmission of the "HTTP get" command from the 
client to the SGSN and the time for the first data packet to be transferred from the SGSN to the client. These times can, 
for example, be estimated as follows: 

• When the first TCP data packet is sent from the SGSN towards the terminal (point 2), note the time and TCP 
sequence number for this and the following TCP packets. 

• When a TCP ACK is received from the terminal (not shown in figure), note the time of arrival of this packet. 

• Calculate the time between the reception of the TCP ACK packet and the time of sending of the last TCP 
packet which was acknowledged in the TCP ACK. 

By adding the two times (SGSN to server and SGSN to terminal) an estimate of the total response time can be 
calculated. 

Note that if packets are lost the calculated ACK time can be larger than the shortest possible time. This can, for 
instance, be handled by making several ACK time measurements and using the shortest or the average time. 



B.4 Data Rate Parameters 

In this example the HTTP Mean Data Rate QoS Parameter is measured in the SGSN node interface for a HTTP 
download of a single file from an HTTP server. The steps involved in the measurement are: 

• The HTTP session is set up between the terminal (client) and the HTTP server. 

• At a standardized interface at the SGSN the data packets for the HTTP session are inspected and the HTTP 
and TCP protocol headers are analyzed. 

• When the first data packet containing content is received at the SGSN interface (the PCO and POR) the time 
of arrival is recorded. This is represented by point 2 in the sequence diagram for the SGSN node in Figure B.3. 

• For the whole file transfer from the server to the client the total amount of user data transferred is measured. 
When the file has been completely transferred the information about the total file size is known at the SGSN 
interface. 

• When the last data packet containing content is received at the SGSN interface the time of arrival for that 
packet is recorded (point 3). 
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The Mean Data Rate QoS parameter can now be calculated using the amount of user data transferred, and the 
time between arrival of the first and last data packet containing content. Note that throughput can be calculated 
taking or not taking TCP retransmissions into account. Throughput can also be calculated taking only the 
payload into account or including both pay load and transport protocol headers. 
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Annex C (informative): 
3GPP SA5 "UE IVIanagement" 



The 3GPP SA5 standardization group is discussing opening a new work item for "UE Management". One of the 
proposed purposes of this work item is: 

"Defining a candidate set of UE measurements to enhance multi-technology radio network planning as well as self auto- 
optimization where possible, including the consideration of the impact on UE performance and air interface. The UE 
measurements do not imply to be the end-to-end, e.g., end-to-end service related measurements." 

This potential 3 GPP SA5 work item might be related to the QoE reporting concept described in annex A. 
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